convergence factor
Pre-insertion resistors temperature prediction based on improved WOA-SVR
Dai, Honghe, Mo, Site, Wang, Haoxin, Yin, Nan, Fan, Songhai, Li, Bixiong
The pre-insertion resistors (PIR) within high-voltage circuit breakers are critical components and warm up by generating Joule heat when an electric current flows through them. Elevated temperature can lead to temporary closure failure and, in severe cases, the rupture of PIR. To accurately predict the temperature of PIR, this study combines finite element simulation techniques with Support Vector Regression (SVR) optimized by an Improved Whale Optimization Algorithm (IWOA) approach. The IWOA includes Tent mapping, a convergence factor based on the sigmoid function, and the Ornstein-Uhlenbeck variation strategy. The IWOA-SVR model is compared with the SSA-SVR and WOA-SVR. The results reveal that the prediction accuracies of the IWOA-SVR model were 90.2% and 81.5% (above 100$^\circ$C) in the 3$^\circ$C temperature deviation range and 96.3% and 93.4% (above 100$^\circ$C) in the 4$^\circ$C temperature deviation range, surpassing the performance of the comparative models. This research demonstrates the method proposed can realize the online monitoring of the temperature of the PIR, which can effectively prevent thermal faults PIR and provide a basis for the opening and closing of the circuit breaker within a short period.
Improved Convergence Rates of Anderson Acceleration for a Large Class of Fixed-Point Iterations
Garner, Casey, Lerman, Gilad, Zhang, Teng
This paper studies Anderson acceleration (AA) for fixed-point methods ${x}^{(k+1)}=q({x}^{(k)})$. It provides the first proof that when the operator $q$ is linear and symmetric, AA improves the root-linear convergence factor over the fixed-point iterations. When $q$ is nonlinear, yet has a symmetric Jacobian at the solution, a slightly modified AA algorithm is proved to have an analogous root-linear convergence factor improvement over fixed-point iterations. Simulations verify our observations. Furthermore, experiments with different data models demonstrate AA is significantly superior to the standard fixed-point methods for Tyler's M-estimation.
Anderson Acceleration as a Krylov Method with Application to Asymptotic Convergence Analysis
De Sterck, Hans, He, Yunhui, Krzysik, Oliver A.
Anderson acceleration (AA) is widely used for accelerating the convergence of nonlinear fixed-point methods $x_{k+1}=q(x_{k})$, $x_k \in \mathbb{R}^n$, but little is known about how to quantify the convergence acceleration provided by AA. As a roadway towards gaining more understanding of convergence acceleration by AA, we study AA($m$), i.e., Anderson acceleration with finite window size $m$, applied to the case of linear fixed-point iterations $x_{k+1}=M x_{k}+b$. We write AA($m$) as a Krylov method with polynomial residual update formulas, and derive recurrence relations for the AA($m$) polynomials. Writing AA($m$) as a Krylov method immediately implies that $k$ iterations of AA($m$) cannot produce a smaller residual than $k$ iterations of GMRES without restart (but without implying anything about the relative convergence speed of (windowed) AA($m$) versus restarted GMRES($m$)). We find that the AA($m$) residual polynomials observe a periodic memory effect where increasing powers of the error iteration matrix $M$ act on the initial residual as the iteration number increases. We derive several further results based on these polynomial residual update formulas, including orthogonality relations, a lower bound on the AA(1) acceleration coefficient $\beta_k$, and explicit nonlinear recursions for the AA(1) residuals and residual polynomials that do not include the acceleration coefficient $\beta_k$. Using these recurrence relations we also derive new residual convergence bounds for AA(1) in the linear case, demonstrating how the per-iteration residual reduction $||r_{k+1}||/||r_{k}||$ depends strongly on the residual reduction in the previous iteration and on the angle between the prior residual vectors $r_k$ and $r_{k-1}$. We apply these results to study the influence of the initial guess on the asymptotic convergence factor of AA(1), and to study AA(1) residual convergence patterns.
Multi-Power Level $Q$-Learning Algorithm for Random Access in NOMA mMTC Systems
Silva, Giovanni Maciel Ferreira, Abrão, Taufik
The massive machine-type communications (mMTC) service will be part of new services planned to integrate the fifth generation of wireless communication (B5G). In this scenario, the massive random access (RA) problem arises when two or more devices collide when selecting the same resource block. There are several techniques to deal with this problem. One of them deploys Q-learning (QL), in which devices store in their Q-table the rewards sent by the central node that indicate the quality of the transmission performed. The device learns the best resource blocks to select and transmit to avoid collisions. The numerical results reveal that the best performance-complexity trade-off is obtained by using a higher number of power levels, typically eight levels. The proposed MPL-QL can deliver better throughput and lower latency compared to other recent QL-based algorithms found in the literature.
Fast model averaging via buffered states and first-order accelerated optimization algorithms
Esteki, Amir-Salar, Moradian, Hossein, Kia, Solmaz S.
In this letter, we study the problem of accelerating reaching average consensus over connected graphs in a discrete-time communication setting. Literature has shown that consensus algorithms can be accelerated by increasing the graph connectivity or optimizing the weights agents place on the information received from their neighbors. Here, instead of altering the communication graph, we investigate two methods that use buffered states to accelerate reaching average consensus over a given graph. In the first method, we study how convergence rate of the well-known first-order Laplacian average consensus algorithm changes when agreement feedback is generated from buffered states. For this study, we obtain a sufficient condition on the ranges of buffered state that leads to faster convergence. In the second proposed method, we show how the average consensus problem can be cast as a convex optimization problem and solved by first-order accelerated optimization algorithms for strongly-convex cost functions. We construct an accelerated average consensus algorithm using the so-called Triple Momentum optimization algorithm. The first approach requires less global knowledge for choosing the step size, whereas the second one converges faster in our numerical results by using extra information from the graph topology. We demonstrate our results by implementing the proposed algorithms in a Gaussian Mixture Model (GMM) estimation problem used in sensor networks.
On the Suboptimality of Negative Momentum for Minimax Optimization
Smooth game optimization has recently attracted great interest in machine learning as it generalizes the single-objective optimization paradigm. However, game dynamics is more complex due to the interaction between different players and is therefore fundamentally different from minimization, posing new challenges for algorithm design. Notably, it has been shown that negative momentum is preferred due to its ability to reduce oscillation in game dynamics. Nevertheless, existing analysis about negative momentum was restricted to simple bilinear games. In this paper, we extend the analysis to smooth and strongly-convex strongly-concave minimax games by taking the variational inequality formulation. By connecting momentum method with Chebyshev polynomials, we show that negative momentum accelerates convergence of game dynamics locally, though with a suboptimal rate. To the best of our knowledge, this is the \emph{first work} that provides an explicit convergence rate for negative momentum in this setting.